Numerical Solution of Implicitly Constrained Optimization Problems ∗

Many applications require the minimization of a smooth function f : R nu → R whose evaluation requires the solution of a system of nonlinear equations. This system represents a numerical simulation that must be run to evaluate f. This system of nonlinear equations is referred to as an implicit constraint. In principle f can be minimized using the steepest descent method or Newton-type methods for unconstrained minimization. However, for the practical application of derivative based methods for the minimization of f one has to deal with many interesting issues that arise out of the presence of the implicit constraints that must be solved to evaluate f. This article studies some of these issues, ranging from sensitivity and adjoint techniques for derivative computation to implementation issues in Newton-type methods. A discretized optimal control problem governed by the unsteady Burgers equation is used to illustrate the ideas. The material in this article is accessible to anyone with knowledge of Newton-type methods for finite dimensional unconstrained optimization. Many of the concepts discussed in this article extend to and are used in areas such as optimal control and PDE constrained optimization. 1. Introducton. We are interested in the solution of min u∈U f (u), (1.1) where U is a closed convex subset of R n u , such as U = R n u or U = [−1, 1] n u , and f : U → R is a smooth function. The numerical solution of (2.1) using gradient-based and Newton-type methods is discussed in most courses on Numerical Analysis (at least for the case U = R n u) and in courses on Optimization. Many textbooks such as [12, 22, 26] provide an excellent introduction into these methods. We investigate their application in the case where the evaluation of objective function f requires the solution of a system of nonlinear equations. This situation arises in many science and engineering applications in which the evaluation of the objective function involves a simulation. We refer to the system of nonlinear equations (the simulation) as an implicit constraint. In theory standard optimization algorithms, such as those discussed in the textbooks [12, 22, 26] can be applied to the solution of (1.1). However, the practical application of these methods quickly leads to interesting questions related to • gradient and Hessian computations for objective functions f whose evaluation involves the solution of an implicit constraint, • software …